2 Classical Parametric Inference

It is convenient to have an abstract framework for discussing statistical theory. The general problem is that there exists an unknown parameter θ0, which we wish to find out about. To have something concrete in mind, consider for example a population with the N(θ0 , 1) distribution, where θ0 is an unknown constant. If we do not have any a priori information about θ0 then it stands to reason that we consider every distribution of the form N(θ , 1), as θ ranges over R, and then use data to make inference about the real, unknown θ0. The general framework is this: We have a parameter space Θ and the real θ0 is in Θ, but we do not its value. For every θ ∈ Θ, let Pθ denote the underlying probability, which is computed by assuming that θ0 = θ. Similarly define Eθ, Varθ, Covθ, etc. Then, the idea is to take a sample—typically an independent sample—X = (X1 , . . . , Xn)—from Pθ0 . If the true (unknown) θ0 were equal to some (known) θ1 ∈ Θ, then one would expect X to behave like an independent sample from Pθ1 . If so, then we declare that θ0 might well be θ0. Else, we reject the notion that θ0 = θ1. The remainder of these notes make this technique precise in more special settings.